{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "881bde60",
   "metadata": {},
   "source": [
    "# 快速成为深度学习全栈工程师第3课书面作业\n",
    "\n",
    "学号：114499  \n",
    "\n",
    "**作业内容：**  \n",
    "1. 请用代码表示出构建TensorFlow模型的几个关键步骤  \n",
    "2. 请用代码表示出构建PyTorch模型的几个关键步骤"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26437193",
   "metadata": {},
   "source": [
    "## 作业1：用代码表示出构建TensorFlow模型的几个关键步骤"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "838e9339",
   "metadata": {},
   "source": [
    "### 步骤1：导入tensorflow库"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e292a008",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "ade6ae82",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.7.0\n"
     ]
    }
   ],
   "source": [
    "print(tf.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "292d9298",
   "metadata": {},
   "source": [
    "可以看出本机的tensorflow为2.7.0版本，与课程上的tensorflow 1.x版本有较大差别，下面主要以tensorflow 2.x模型来描述关键步骤。\n",
    "\n",
    "\n",
    "### 步骤2：定义训练及测试数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "9d6eb966",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x_train shape: (60000, 28, 28, 1)\n",
      "60000 train samples\n",
      "10000 test samples\n"
     ]
    }
   ],
   "source": [
    "# Model / data parameters\n",
    "num_classes = 10\n",
    "input_shape = (28, 28, 1)\n",
    "\n",
    "# the data, split between train and test sets\n",
    "(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()\n",
    "\n",
    "# Scale images to the [0, 1] range\n",
    "x_train = x_train.astype(\"float32\") / 255\n",
    "x_test = x_test.astype(\"float32\") / 255\n",
    "# Make sure images have shape (28, 28, 1)\n",
    "x_train = np.expand_dims(x_train, -1)\n",
    "x_test = np.expand_dims(x_test, -1)\n",
    "print(\"x_train shape:\", x_train.shape)\n",
    "print(x_train.shape[0], \"train samples\")\n",
    "print(x_test.shape[0], \"test samples\")\n",
    "\n",
    "\n",
    "# convert class vectors to binary class matrices\n",
    "y_train = tf.keras.utils.to_categorical(y_train, num_classes)\n",
    "y_test = tf.keras.utils.to_categorical(y_test, num_classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "009f9a90",
   "metadata": {},
   "source": [
    "### 步骤3：定义神经网络模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "8e86a531",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " input_1 (InputLayer)        [(None, 28, 28, 1)]       0         \n",
      "                                                                 \n",
      " conv2d (Conv2D)             (None, 26, 26, 32)        320       \n",
      "                                                                 \n",
      " max_pooling2d (MaxPooling2D  (None, 13, 13, 32)       0         \n",
      " )                                                               \n",
      "                                                                 \n",
      " conv2d_1 (Conv2D)           (None, 11, 11, 64)        18496     \n",
      "                                                                 \n",
      " max_pooling2d_1 (MaxPooling  (None, 5, 5, 64)         0         \n",
      " 2D)                                                             \n",
      "                                                                 \n",
      " flatten (Flatten)           (None, 1600)              0         \n",
      "                                                                 \n",
      " dropout (Dropout)           (None, 1600)              0         \n",
      "                                                                 \n",
      " dense (Dense)               (None, 10)                16010     \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 34,826\n",
      "Trainable params: 34,826\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "inputs = tf.keras.Input(shape=input_shape)\n",
    "x = tf.keras.layers.Conv2D(32, kernel_size=(3, 3), activation=\"relu\")(inputs)\n",
    "x = tf.keras.layers.MaxPooling2D(pool_size=(2, 2))(x)\n",
    "x = tf.keras.layers.Conv2D(64, kernel_size=(3, 3), activation=\"relu\")(x)\n",
    "x = tf.keras.layers.MaxPooling2D(pool_size=(2, 2))(x)\n",
    "x = tf.keras.layers.Flatten()(x)\n",
    "x = tf.keras.layers.Dropout(0.5)(x)\n",
    "outputs = tf.keras.layers.Dense(num_classes, activation=\"softmax\")(x)\n",
    "model = tf.keras.Model(inputs=inputs, outputs=outputs)\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6dd956d1",
   "metadata": {},
   "source": [
    "### 步骤4：模型训练\n",
    "* 定义损失函数：这里用的是分类交叉熵函数\n",
    "* 指定优化器：这里用的是adam\n",
    "* 指定度量指标：这里是accuray，即准确度\n",
    "* 指定批次：即batch_size，即每次训练送入多少数据，这里是128\n",
    "* 指定迭代次数：即epoch，一批样本可以分成多个批次送入训练完成一个epoch训练，总共做几个epoch。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "6c4673f4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/15\n",
      "422/422 [==============================] - 15s 34ms/step - loss: 0.3606 - accuracy: 0.8889 - val_loss: 0.0803 - val_accuracy: 0.9780\n",
      "Epoch 2/15\n",
      "422/422 [==============================] - 14s 34ms/step - loss: 0.1098 - accuracy: 0.9665 - val_loss: 0.0579 - val_accuracy: 0.9837\n",
      "Epoch 3/15\n",
      "422/422 [==============================] - 14s 34ms/step - loss: 0.0838 - accuracy: 0.9737 - val_loss: 0.0475 - val_accuracy: 0.9878\n",
      "Epoch 4/15\n",
      "422/422 [==============================] - 15s 35ms/step - loss: 0.0717 - accuracy: 0.9781 - val_loss: 0.0412 - val_accuracy: 0.9873\n",
      "Epoch 5/15\n",
      "422/422 [==============================] - 15s 35ms/step - loss: 0.0619 - accuracy: 0.9811 - val_loss: 0.0381 - val_accuracy: 0.9900\n",
      "Epoch 6/15\n",
      "422/422 [==============================] - 14s 34ms/step - loss: 0.0528 - accuracy: 0.9835 - val_loss: 0.0371 - val_accuracy: 0.9897\n",
      "Epoch 7/15\n",
      "422/422 [==============================] - 14s 34ms/step - loss: 0.0500 - accuracy: 0.9842 - val_loss: 0.0347 - val_accuracy: 0.9903\n",
      "Epoch 8/15\n",
      "422/422 [==============================] - 14s 34ms/step - loss: 0.0468 - accuracy: 0.9854 - val_loss: 0.0306 - val_accuracy: 0.9922\n",
      "Epoch 9/15\n",
      "422/422 [==============================] - 14s 34ms/step - loss: 0.0428 - accuracy: 0.9858 - val_loss: 0.0332 - val_accuracy: 0.9910\n",
      "Epoch 10/15\n",
      "422/422 [==============================] - 14s 34ms/step - loss: 0.0403 - accuracy: 0.9868 - val_loss: 0.0304 - val_accuracy: 0.9923\n",
      "Epoch 11/15\n",
      "422/422 [==============================] - 14s 34ms/step - loss: 0.0385 - accuracy: 0.9880 - val_loss: 0.0295 - val_accuracy: 0.9915\n",
      "Epoch 12/15\n",
      "422/422 [==============================] - 15s 34ms/step - loss: 0.0363 - accuracy: 0.9878 - val_loss: 0.0298 - val_accuracy: 0.9913\n",
      "Epoch 13/15\n",
      "422/422 [==============================] - 15s 34ms/step - loss: 0.0337 - accuracy: 0.9891 - val_loss: 0.0312 - val_accuracy: 0.9905\n",
      "Epoch 14/15\n",
      "422/422 [==============================] - 15s 35ms/step - loss: 0.0334 - accuracy: 0.9893 - val_loss: 0.0273 - val_accuracy: 0.9928\n",
      "Epoch 15/15\n",
      "422/422 [==============================] - 15s 35ms/step - loss: 0.0317 - accuracy: 0.9897 - val_loss: 0.0291 - val_accuracy: 0.9922\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x2460eab1790>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "batch_size = 128\n",
    "epochs = 15\n",
    "\n",
    "model.compile(loss=\"categorical_crossentropy\", optimizer=\"adam\", metrics=[\"accuracy\"])\n",
    "\n",
    "model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, validation_split=0.1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46cdf09e",
   "metadata": {},
   "source": [
    "### 步骤5：用训练结果进行预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "e6068e4f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test loss: 0.020912496373057365\n",
      "Test accuracy: 0.993399977684021\n"
     ]
    }
   ],
   "source": [
    "score = model.evaluate(x_test, y_test, verbose=0)\n",
    "print(\"Test loss:\", score[0])\n",
    "print(\"Test accuracy:\", score[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05415dec",
   "metadata": {},
   "source": [
    "## 作业2：用代码表示出构建PyTorch模型的几个关键步骤"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78b382a3",
   "metadata": {},
   "source": [
    "### 步骤1：导入PyTorch库"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "5f459c0d",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.optim as optim\n",
    "from torchvision import datasets, transforms\n",
    "from torch.utils.data import DataLoader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "f00337f5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.10.0+cpu\n"
     ]
    }
   ],
   "source": [
    "print(torch.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b38752ff",
   "metadata": {},
   "source": [
    "### 步骤2：定义数据集\n",
    "PyTorch中数据集必须继承于Dataset类，这里通过PyTorch预置的MNIST数据集，获取数据，同时实例化训练与测试数据集对象。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "085f9762",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 下载训练集\n",
    "train_dataset = datasets.MNIST(root='./',\n",
    "                train=True,\n",
    "                transform=transforms.ToTensor(),\n",
    "                download=True)\n",
    "# 下载测试集\n",
    "test_dataset = datasets.MNIST(root='./',\n",
    "               train=False,\n",
    "               transform=transforms.ToTensor(),\n",
    "               download=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b2bc8bd",
   "metadata": {},
   "source": [
    "### 步骤3：定义数据装载器"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "17b846d6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 批次大小\n",
    "batch_size = 64\n",
    "\n",
    "# 装载训练集\n",
    "train_loader = DataLoader(dataset=train_dataset,\n",
    "                      batch_size=batch_size,\n",
    "                      shuffle=True)\n",
    "# 装载测试集\n",
    "test_loader = DataLoader(dataset=test_dataset,\n",
    "                     batch_size=batch_size,\n",
    "                     shuffle=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "20108314",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([64, 1, 28, 28])\n",
      "torch.Size([64])\n"
     ]
    }
   ],
   "source": [
    "for i, data in enumerate(train_loader):\n",
    "    # 获得数据和对应的标签\n",
    "    inputs, labels = data\n",
    "    print(inputs.shape)\n",
    "    print(labels.shape)\n",
    "    break"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5421b02b",
   "metadata": {},
   "source": [
    "### 步骤4：定义神经网络模型\n",
    "* 通过初始化函数定义定义网络的静态结构组件。  \n",
    "* 通过forward函数，定义网络向前计算步骤，即从输入如何得到输出。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "a59c9e09",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 定义网络结构\n",
    "class Net(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "        self.conv1 = nn.Sequential(nn.Conv2d(1, 32, 5, 1, 2), nn.ReLU(), nn.MaxPool2d(2, 2))\n",
    "        self.conv2 = nn.Sequential(nn.Conv2d(32, 64, 5, 1, 2), nn.ReLU(), nn.MaxPool2d(2, 2))\n",
    "        self.fc1 = nn.Sequential(nn.Linear(64 * 7 * 7, 1000), nn.Dropout(p=0.4), nn.ReLU())\n",
    "        self.fc2 = nn.Sequential(nn.Linear(1000, 10), nn.Softmax(dim=1))\n",
    "        \n",
    "    def forward(self, x):\n",
    "        # ([64, 1, 28, 28])\n",
    "        x = self.conv1(x)\n",
    "        x = self.conv2(x)\n",
    "        x = x.view(x.size()[0], -1)\n",
    "        x = self.fc1(x)\n",
    "        x = self.fc2(x)\n",
    "        return x"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22dbdce5",
   "metadata": {},
   "source": [
    "### 步骤5：定义损失函数及优化器\n",
    "* 损失函数：这里用交叉熵函数  \n",
    "* 优化器：这里也采用adam优化器"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "5b7606c8",
   "metadata": {},
   "outputs": [],
   "source": [
    "LR = 0.0003\n",
    "# 定义模型\n",
    "model = Net()\n",
    "# 定义代价函数\n",
    "entropy_loss = nn.CrossEntropyLoss()\n",
    "# 定义优化器\n",
    "optimizer = optim.Adam(model.parameters(), LR)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5bdd780a",
   "metadata": {},
   "source": [
    "### 步骤6：训练及测试模型\n",
    "#### 训练 \n",
    "* 对于训练模型，需要在之前调用model.train()函数，这样backward计算时会自动计算梯度并反向传播  \n",
    "* 针对一个批次：先正向计算结果，比较结果的目标的损失，再反向计算梯度，再更新权值，再下一个批次。  \n",
    "* 针对下一个epoch，重复上面过程。\n",
    "\n",
    "#### 测试\n",
    "* 对于测试模型，需要在之前调用model.eval()函数，这样不会再更新网络内部的权值。  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "92f927fa",
   "metadata": {},
   "outputs": [],
   "source": [
    "def train():\n",
    "    model.train()\n",
    "    for i, data in enumerate(train_loader):\n",
    "        # 获得数据和对应的标签\n",
    "        inputs, labels = data\n",
    "        # 获得模型预测结果，（64，10）\n",
    "        out = model(inputs)\n",
    "        # 交叉熵代价函数out(batch,C),labels(batch)\n",
    "        loss = entropy_loss(out, labels)\n",
    "        # 梯度清0\n",
    "        optimizer.zero_grad()\n",
    "        # 计算梯度\n",
    "        loss.backward()\n",
    "        # 修改权值\n",
    "        optimizer.step()\n",
    "\n",
    "\n",
    "def test():\n",
    "    model.eval()\n",
    "    correct = 0\n",
    "    for i, data in enumerate(test_loader):\n",
    "        # 获得数据和对应的标签\n",
    "        inputs, labels = data\n",
    "        # 获得模型预测结果\n",
    "        out = model(inputs)\n",
    "        # 获得最大值，以及最大值所在的位置\n",
    "        _, predicted = torch.max(out, 1)\n",
    "        # 预测正确的数量\n",
    "        correct += (predicted == labels).sum()\n",
    "    print(\"Test acc: {0}\".format(correct.item() / len(test_dataset)))\n",
    "    \n",
    "    correct = 0\n",
    "    for i, data in enumerate(train_loader):\n",
    "        # 获得数据和对应的标签\n",
    "        inputs, labels = data\n",
    "        # 获得模型预测结果\n",
    "        out = model(inputs)\n",
    "        # 获得最大值，以及最大值所在的位置\n",
    "        _, predicted = torch.max(out, 1)\n",
    "        # 预测正确的数量\n",
    "        correct += (predicted == labels).sum()\n",
    "    print(\"Train acc: {0}\".format(correct.item() / len(train_dataset)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "613dc017",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch: 0\n",
      "Test acc: 0.9764\n",
      "Train acc: 0.9774166666666667\n",
      "epoch: 1\n",
      "Test acc: 0.9846\n",
      "Train acc: 0.9857166666666667\n",
      "epoch: 2\n",
      "Test acc: 0.9887\n",
      "Train acc: 0.9893333333333333\n",
      "epoch: 3\n",
      "Test acc: 0.9897\n",
      "Train acc: 0.9904666666666667\n",
      "epoch: 4\n",
      "Test acc: 0.9884\n",
      "Train acc: 0.9893166666666666\n",
      "epoch: 5\n",
      "Test acc: 0.9874\n",
      "Train acc: 0.9895166666666667\n",
      "epoch: 6\n",
      "Test acc: 0.9906\n",
      "Train acc: 0.9934666666666667\n",
      "epoch: 7\n",
      "Test acc: 0.9858\n",
      "Train acc: 0.9895833333333334\n",
      "epoch: 8\n",
      "Test acc: 0.9911\n",
      "Train acc: 0.9942\n",
      "epoch: 9\n",
      "Test acc: 0.9906\n",
      "Train acc: 0.9933666666666666\n",
      "epoch: 10\n",
      "Test acc: 0.9911\n",
      "Train acc: 0.9953166666666666\n",
      "epoch: 11\n",
      "Test acc: 0.9916\n",
      "Train acc: 0.9952666666666666\n",
      "epoch: 12\n",
      "Test acc: 0.9917\n",
      "Train acc: 0.9959\n",
      "epoch: 13\n",
      "Test acc: 0.9917\n",
      "Train acc: 0.9960833333333333\n",
      "epoch: 14\n",
      "Test acc: 0.9915\n",
      "Train acc: 0.9964166666666666\n",
      "epoch: 15\n",
      "Test acc: 0.9915\n",
      "Train acc: 0.9957333333333334\n",
      "epoch: 16\n",
      "Test acc: 0.9916\n",
      "Train acc: 0.99665\n",
      "epoch: 17\n",
      "Test acc: 0.9926\n",
      "Train acc: 0.99685\n",
      "epoch: 18\n",
      "Test acc: 0.9923\n",
      "Train acc: 0.9972\n",
      "epoch: 19\n",
      "Test acc: 0.9922\n",
      "Train acc: 0.9971666666666666\n"
     ]
    }
   ],
   "source": [
    "for epoch in range(0, 20):\n",
    "    print('epoch:',epoch)\n",
    "    train()\n",
    "    test()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "656511f6",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
